Overview of the MPEG-7 Standard and of Future Challenges for Visual Information Analysis

The goal of the MPEG-7 standard is to allow interoperable searching, indexing, filtering, and access of audio-visual (AV) content by enabling interoperability among devices and applications that deal with AV content description. MPEG-7 specifies the description of features related to the AV content as well as information related to the management of AV content. As illustrated in Figure 1, the scope of the standard is to define the representation of the description, that is, the syntax and the semantics of the structures used to create MPEG-7 descriptions. For most description tools, the standard does not provide normative tools for the generation nor for the consumption of the description. This is not necessary to guarantee interoperability and, moreover, this allows future improvements to be included in MPEG-7 compliant applications. However, as will be discussed in this paper, in order to guarantee interoperability for some lowlevel features, MPEG-7 also specifies part of the extraction process. MPEG-7 descriptions take two possible forms: (1) a textual XML form suitable for editing, searching, filtering, and browsing and (2) a binary form suitable for storage, transmission, and streaming. Overall, the standard specifies four types of normative elements illustrated in Figure 2: Descriptors, Description Schemes (DSs), a Description Definition Language (DDL), and coding schemes. In order to describe AV content, a set of Descriptors has to be used. In MPEG-7, a Descriptor defines the syntax and the semantics of an elementary feature. A Descriptor can deal with low-level features, which represent the signal characteristics, such as color, texture, shape, motion, audio energy or Description generation Description consumption Description